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ABSTRACT 

The rationale for participant observation call for a greater reliance 
oerience and training of observers and on systematic profcedures for 
and inter-class comparisons s, than on the development of a system for directly and 
reliably recording categories or signs of behavioral fragments. Variations in 
Caching and in observation must be analysed as interdependent sources which both 
contribute meaningful descriptions of differences between classes. Recording 
samples of observed behaviors is essential for training and analysis o 

A«olications using teams of observers in Head Start and inner cit> and suburban 
elementLy school classes are described and discussed with reference to methodology 
and data Lduction. Films were made of a stratified sample of classes in 
anchor observational reports and ratings and for the purpose of providing p mary 
data on stylistic variation across school location and grade level. 
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INTRODUCTION 

All too often educational studies employ a single recording technique to abstract 
teacher behavior into data. The monolith is this singular strategy rather than the 
claims and procedures of any one school of observational thought. Such a criticism is 
not confined to educational research, but to any studies that focus on complex human 
behaviors for which there is no optimal methodology that is accepted by professional 
consensus as being the epitome of validity. Although a particular methodological 
approach - participant observation (Bruyn, i966) - will be described, the discussion 
of perspective is crucial to its elaboration. The vehicle of inference for participant 
observation is "observer" with experience, training, and theory rather than rating scale, 
checklist or behavioral protocal. In order to comprehend the validity of any of these 
vehitles it is necessary to explore their potential diverse contributions and to carefully 
describe defects in Instrumentation, methodology and substance. 

Participant observation is not cast as the only or preferred approach, but rather 
as a necessary component of research activity that aims at inferring useful data from 
teacher behaviors. The fact that such a strategy does not result in easily reportable 
and grossly comparable data should not be a deterrrent to its use if there is reason to 
believe that the behavior being studied is so diverse and complex that descriptive 
problems are inherent because of this diversity and complexity. Social sciences (and 
Other sciences, as well) always run the rish of reporting that which is easy to describe 
rather than that which is important to the phenomena being studied. 



RATIONALE 

Strategies for obtaining data on teacher variation cover a wide range of procedures. 
Quantification is variously based on rating scales, behavioral categories, checklists, 
interaction analyses and projective inferences. Reliability is more a question 
of definition of behavioral units than of their relevance to teacher effectiveness. 

The Substance of the behavior that is designated by the observational model 

is a reflection of either the instrument maker’s or the observer’s bias. THiichever 



! "The research reported herein was performed pursuant to a contract .with the Office 
of Economic Opportunity, Executive Office of the President, Washington, D.C., 
i!C506. The opinions expressed herein are those of the author and should not be 
construed as representing the opinions or policy of any agency of the United States 
Government . ” 
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Is the case, there is always a presumption about educational goals and effective 
Impl^entation. This is Just as true of rating scales as f 
must make a orior decision about what is to be observed. It is not clear that 
anrextant system is based on a theory which would systematically direct us to study 
particular behavioral categories. 



When explicit attempts are made to empirically judge eff(Bctiveness by observing 
changes in children during the time they are with a particular ^ 

more, to select units of teacher behavior because of their relation to 
are karls because teacher effects are engulfed by developmental and f 

effects and also, and perhaps more significantly, the behaviors that are most directly 
affected by teachers are not easily defined or measured.. Achievement tests 8 
abstraction of intellectual behavior which may very well be invariant to teacher 
effects, especially when compared to intelligence and social class variance. Tnis 
is not to imply that there are no teacher effects, but only that given the instru- 
ments and variables conventionally used, for practical purposes, they are not 
measurable, at least .with the samples of teachers and children that have been used 
in teacher effectiveness researchc This is an important at least for, as has been 
pointed out in psychotherapy research, demonstrated effectiveness of a particular 
therapist or procedure is very much a function of the diagnosis and severity of the 
patient. It is possible and probably that teacher effectiveness studies must take 
into careful consideration the age, sex, and educational-intellectual status of 
students. The teacher variable will probably prove to be more demonstrably effective 
for disadvantaged, disturbed, retarded and generally disabled children than for 
normal children because the variability in criteria is, to a large egree, accounte 
for by Independent variables that are constructively and methodologically highly 
correlated. There is a confounding between the research problem - are teachers 
differentially effective? - and the measurement problem, that is largely unresolved. 



While admitting that the ultimate criteria of teacher effectiveness are changes 
in children, it does not necessarily follow that the Important teacher variable (or 
variables) should be derived by regressing chapfees (in children) against a myriad 
of input variables (teacher behaviors) . For this to be the recommended procedure 
it would have to be established that the criteria are desirable and that they are 
meaningfully linked to teacher behaviors, neither of which is definitively so. Research 
on teaching is faced with a forbidding gap between teaching and learning which is 
partly a function of the autonomy of teachers and partly of the nature and limitations 
of teaching and measurement technology. 

Failure to develop a predictive system for determining effectiveness has been 
accompanied by (and partly by default led to) the development of authoritative 
systems whereby one or more professioual.'J describe what makes an effective teacher. 

Items, scales or categories are abstracted so that they can be used by a more or ^ 
less skilled observer, to obtain data on the purported effectiveness of a sample jr 
teachers. Behavioral units can be quite global,, encompassing such broad areas as 
permissiveness, warmth, creativity or control, or they can be extremely specific 
relatively nonjudgemental, such as recording the number of times or amount of time 
that particular behaviors and interactions take place. Global assessment depends 
on trained and experienced observers while specific assessment depends on trained b'*t 
not necessarily experienced observers (experience referring to teaching and training 
referring to observer training) . 
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The construct validity of any more or less global or specfic system will depend 
on not only the substance of categories or itemp» but on other dlsideratS as well. 

In fact} substance might very well be of least significiance in light of situational 
and procedural varibllltles that are often erronepusly 'assumed to be relatively constant. 
Given the fact that teachers vary, it does not necessarily follow that procedures are 
directly comparable, operational goals are the same, samples of children in different 
classes require the same approach, curricular and time of day variations are insignifi- 
cant or cultural forces or particular schools are not predisposing. When the burden 
is on the Instrument (rather than the observer) it is difficult or impossible to 
correct for conf tending that is implicit in each of these sources of variation. Given 
Instruments, will only be effective to the extent that these intervening variables are 
not only controlled for (presumably by randomization or manipulation) but are measured 
and, it follows, whose distributions are adquately represented in the given sample of 
classes. This suggests that either studies of teaching should concentrate on in- 
tensive surveys of relatively homogenious clusters of classes that differ on few but 
potent dimensions, or that large scale studies include manipulation of curricular,' 
sampling of children, in-service training and supervision. This is to say that there 
is too much noise in the system for any single instrument to validly assess teacher 
effectiveness. This is just as true if the Instrument is based on a construct as it 
is if it has been empirically derived. 

Another rather imposing source of variation is the observer both-|-the procedurfcs 
by which he is trained and those that he uses in the course of his observations. It 
is not only that different people see different things, but that the conditions of 
training, visiting classes, feedback, and articulation cannot be assumed to be constant. 
The use of a single instrument will not Insure comparable data unless either the 
observational process in continuously standardized, the instrument has built in 
features which suppress observer and observing contamination, additional data is col- 
lected to provide for necessary nominal distinctions, or the variability In phenomena 
being observed dominated observer variability in a direction consonant with the 
purpose of the data gathering process. 

It follows that no single strategy is Inherently superior to another one but 
that there are situatipnal, temporal, economic, and personnel considerations which 
will suggest that one approach will be' more valid than another. The reduction of 
teaching behavior is desirable because inference is based on more clearly understood 
judgements. However,- reduction can lead to spurious and often misleading data, if it 
is not accompanied by compatib.la' reduction of other relevant behaviors of teachers, 
children, and schools. Furthermore, the sin quo non of reduction is that the trans- 
formation be reversible. If reduction leads to a collection of irreversible bits that 
cannot be associated with the child's and teacher's other (and more global) behaviors, 
then studies of teaching will leave the domain of education and enter some other 
(possibly meaningful) domain. There are obviously impelling reasons why teaching should 
be validly assessed, not the least of which is upgrading teacher education, gaining 
insights into learning of both teachers and children and studying social interactions . 

If reductlonlsra leads away from these by so abstracting and fragmenting behavior then 
It is likely that it will contribute much more to behavioral analysis than to change. 

The greater the reduction to highly reliable bits of teacher behavior ,• the more 
likely it Is that accurate predictions will be made of correspondingly reduced to bits 
of child behavior. Therefore, if the research goal is to get such correspondence, dis- 
regarding its relevance for teaching and learning, then maximal reduction is to be 
desired. But the reductidn process, in general, ignores relevance and only accidentally 
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Ikrovides Indices for units of behavior that are clinically meaningful. Human behavior 
has not been structured (theoretically) as an accumulation of behavioral bits that go 
together in an orderly and linear model. It is not at all clear that ^>.ese bits have 
any useful meaning by themselves. It is a pragmatic question that can be dealt with 
only in tferms of specified applications which become the guage of usefulness. The 
research decision to concentrate on any given units is germane not only to methodo- 
logical considerations — how is the unit best measured? — but to the theoretical con- 
nection between teacher and learner. This connection can be conceptualized as being 
nlapped by any level of obstractlon or generality. The crucial question arises when 
dlirtical requirements demand reversibility — that results under any system of inquiry 
be useful as feedback in order to affect behavior other than that which is under a 
microscope. There is just as much need for transfer from datum to person as there 
is from skill to ability. Without this transfer both systems would be sterile. 

Transfer is implicit in a well ordered and predictable system where reversibility 
(from behavior to abstraction to behavior) is generated from an object (Intra) and 
dcross objects (inter). An individual’s within variability over abilities is re- 
flective of sampling variation across individuals and time and vice versa. The 
Stanford-Binet IQ is reversible (for middle class children) not because we can go 
directly back to the individual from the IQ, but because we can go the sample and 
then, in ^ meaningful way, back tb the Individual. "Meaningful way" refers to the well 
drdered system whereby a probablistic statement can be made about the individual's 
future academic behavior with regards to the group. Without this characteristic test^ 
bcores or observational data become one way streets that make no useful connections. 

Classroom observation is up against the reversibility dilemma no matter how 
Abstract or reliable are the protocols. When data are obtained they may fit into a 
regression, analysis but they cannot be transformed back to the class either directly 
-Ar itidirectly because of the lack of order in the system, either horizcntally or 
Vertically, Because of this, films (or Kineoscopic tapes) are needed to provide a 
ihechanical vehicle for reversibility in the absence of a theoretical or empirical 
Vehicle, Admittedly this only provides for the reversibility; it is not established, 
^ut at least the possibility exists. At the Same time the vehicle for transfer is 
present— various techniques can be applied to the same sample of classrooms. 

Variability of multiple dimensions and strategies can be put to the crude, but im- 
mediate test of viewer (film) variability. Direct comparisons can be made between 
direct recordings of behavioral bits» ratings of qualities, and authoratative 
Judgements, And, most significantly, teachers can be confronted simultaneously 
with data and behavior. .For the present, films would appear to be necessary for the 
development of any form of observational analysis— without films evAn carefully ob- 
tained data will be lost to a specific, non-transferable and irreversible "black 
Isbx" process, 

The fact that the introduction of the photographer or the observer transforms 
the situation is not without theoretical interest. If non-reactive procedures can be 
used in educational studies, as was done by Sexton (1961) and as is recommended by 
Webb^ at al (1966), they are to be desired unless the reactive effects are theoreti- 
cally important in the reconstruction of phenomena being observed. There Is reason to 
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belleve that the principle characteristic of teaching is that it is not observed and 
that feedback is not existent and, in fact, impossible. Education is essentially a 
nonreactive system which is. unaffected by contemporary social movements, recent 
scientific advances and critical reappraisal of current practices. Authorities set up 
the models and pontificate but teachers and principals run the show in autonomous 
conclaves, this autonomy is personal rather than professional. . Textbook and exam- 
ination conformity is obviated by variability along indeterminate and self-defeating 
lines. The model of classroom observer (or photographer) is one that involves more 
than an invasion into the classroom for the convenience of research. It is a different 
and more viable model that permits (but does not insure) a continual reappraisal of 
Curriculum and behavior . The study of unobservable teachers is a paradox without 
resolution. Teaching conceived as art, science, or some combination of the two is 
untenable unless it can be researched on the one hand, or experienced on the other. 

Given the present state of Research technology, the falling tree in the forest does 
not make a sound unless there is someone (or something) to hear it. 

Orchestras need listeners, recorders and critics less they exist in an incestuous 
vacuum. The reinforcement of teachers consists of a bundle of meretricious acts and 
words which contribute more to a religion than a profession and more to a mystical 
epistemology than to a vital language that has some relation to behavior. Therefore, 
the criticism that the observer changes the situation is accepted and encouraged . 

That the necessary research vehicle is just as essential to pedagogy is not a coin- 
cidence. The claim can be made (even if it cannot be rigorously supported) that any 
social scientific techniques should have direct payoff to the individual or groups 
being observed and manipulated. Using film to study teaching is an example of this 

Clslm. 

Disregarding the technique used to record behavior, observational studies are 
usually confronted by comparisons of teaching that depend on values rather than 
behavior. If comparisons are to be made between teachers who lecture and those who 
lead discussions in varying subject fields, any system of measurement will break down 
unless it is either assumed that one approach is inherently better than the other 
(values) or that the different behaviors are irrelevant to the measurement of effec- 
iveness which is to assume that goals transcent methodology. There are several ways 
around this dilemma. The curriculum and/or methodology can be stipulated (Belleck, 
et al 1966) and teaching can be thusly compared. Unless teachers have opportunities 
for patticipation in several manipulations there will be teacher-method confounding. 
Manipulation can be contrived (with or without teacher involvement) or they can be 
unobstrusive (and thus really not manipulations) by selecting sequences of comparable 
behaviors that already exist. In either case and disregarding the observational and 
recording technique, there is some control so that "everything being equal" is not a 
completely empty phrase. 

If manipulations of the first or second kind are impossible to accomplish, ad- 
justments must be made either by restricting the field of study or by using an in— 
strument" that allows for diverse methods, curricular and samples. Such an "instrument 
thight be a series of conditional scales which are selected by the observer depending 
on the curriculum and techniques being used. Comparisons could be made on those scales 
that were selected a sufficlLt number of times. The "Instrument" cculd also be a 
highly trained and experienced team of observers who have necessary skills to compare 
somewhat dissimilar teaching situations. To assume, as is often done, that the observer 
who has the task of selecting and judging, will be more subjective than a series of 
protocols that cannot deal with the complexities of teaching variance, necessarily in- 
volves the tautology that such an observer is definitively subjective, and direct be- 



havioral recording and rating scales are definitively objective. This ^ 

inheritance of the so called "objective" test which is presumed to be objective be- 
cause of its format, not because of its item selection, mode of inquiry or reactive 
ejects! A^it^^di;, the scoring process is less subject to the biases of the scorer 
and the paper and pencil standardization conditions of test administration are 
relatively* *^constant, but this does not provide 

Sellability is an aspect of what might be referred to as , iLtrument 

•Ifl not necessarily primary o It is necessary to consider the effect of the instrument 

bn not only .the subject but the educational process, the selection of • 

of item presentation and the problems Inherent in the transfomation of behavio to 
fifltfl The hieh reliability of "objective" tests is not without a price in external 
subj;ctl!jlty! Thrassumption that reliability is generic to validity has already been 
challenged with regard to "objectivd' testing and it can be similarly challenged with 
tegard to "objective" recording of teaching behavior > 

The argument is the sameo The selection of items and modes of presentation in- 
volves gross subjectivity even though recording and scoring processes (which can be 
onrand the same) are highly reliable. This is not to say that essay tests and the use 
pf the observer-as-instrument necessarily insure external objectivity but only that 
they provide an alternative strategy which can more directly get at higher level 
proLsses. Thinking, reasoning, problem solving and creativity may be vague but they 
come closer to the expressed goals of education than mremorizihg, recalling, and 
educated guessing « Similarly, the assessment of humane, creative, elaborative,n- 
Sightful, and intelligent teaching is more directly to the’ point than counting the 
emount and number of times teachers and students ask questions, make statements, 
make demands, and are silent. This is not to preclude that specifically deflmd be- 
haviors can be Important indicators of generalized functions but. only to gain per- 
spective about their limitat’ons and the value of alternative subjective strategies 
to approach a more profound objectivity than is to be had by using objective 
methods exclusively. 

The question of reaction is not a trivial methodological issue that can be re- 
legated to vagaries of research. The teacher who is "counted and the observer wno is 
counting are part of the system and will respond in some way to this procedure as 
opposed to an alternative one. The reductionlsm involved in counting reduces not 
only behavior, but the work and status of the observer and, therefore, of observational 
process. This is not a polemic for eliminating "counting" but rather an argument 
for questioning any reactive procedure, not because it is reactive, but because of the 
quality and force of the reaction it might evoke. 



> ?ENERAL STATEMENT OF PROCEDURES 

We address ourselves specifically to the problem of evaluating and describing 
the potential effectiveness of teaching in a diverse sample of classrooms and schools 
(or centers). Amount of observation will depend on sample valabillty and 0 sophlstica' 
tlon. In order to obtain approximations of these parameters the design calls for 



2 This followed Campbell and Stanley’s (1963) distinction between Internal and external 

• validity. 



multiple O's making multiple observations of classes over an extended period of time» 
0*8 will have had teaching experience and will participate in seminars prior to and 
throughout the POo Training will consist of a variety of experiences aimed at facili- 
tating inter-0 commtBrication, becoming familiar with a behavioral model and developing 
observation sensitivity. Seminars and workshops prior to PO will be used to screen 
out unsuitable candidates. O’s will participate in an observational seminar where they 
collectivly observe groups of children in classes and discuss at length, teaching and 
learning as they view it« O’s will observe each other teaching children and discuss 
Varieties of approaches and values. 

Films will be utilized in the observational seminar in order to allow for review 
of discussed behaviors at any time. These films should show diverse teachers doing 
aimilar tasks and similar teachers, or a given teacher, functioning in varying ways. 

It id desirable for O’s to view different teachers with the same group of children. 

0*8 will keep careful logs of observed behaviors which will provide detailed 
accounts of teacher, child' and interactional behaviors. Analytical reports will be 
written » utilizing the log as soutces of evidence. Finally, O’s will write inter- 
pretive summaries of teachers and classes, describing their estimation of effect- 
Ivencsa and indicating teaching characterisitics that are critical for their assess- 
ment. Procedures for writing these reports are set forth in greater detail in the 
appendix to this report. 

locales representing important and adequately variable dimensions of teaching 
and child behavior will be constructed in such a way as to relate the observed be- 
havior to the behavioral theory. O’s will Q-sort classes on each of these scales — 
rating all classes on one scale at a time thus minimizing associational biases. O’s 
will underline and label logged behavioral recording according to a notation that 
related scales to specific recorded behaviors. Scaled judgements can then be sup- 
ported by molar sequences of observed and recorded behaviors. 



M001EL 

Although participant observation (PO) varies as to the specific procedures used, 
it is always based on the principle that although the observer (0) will adapt pre- 
conceived structural outlines and dimensional scales on the course of his sumraary, he 
is the instrument for inferring data, rather than any outlines or scales. There must 
he enough intensity and duration in the involvement with the phenomena being studied 
fet it3 unique structure and process to be indentifiable. The amount of contact is 
a function of the kind and degree of distinctions between individuals and agencies that 
are required. Once the target system Is defined 0 has the responsibility of determining 
S traffic pattern for himself which will lead to an understanding of relationships and 
direction. Hypotheses^ are constructed by relating a presumed general theory of behavior 
to the behaviors of the system. PO methodology is Independent of the theory or of the 
w6rking hypotheses— but some articulated theory is necessary. 

6 ie presumed to be experienced and trained although specifications for both de- 
pend on task requirements. Training can be presumed from the previous experience of 0 
or it can take place prior to and during PO. Reliability will depend on the perspective 
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and sensitivity of 0 and multiple 0*s can be used to provide anchoring if diverse 
Situations are to be observed, 0 will observe and become Involved (interviews, utili- 
zation of unobtrusive data, manipulation) to an extent necessary to test hypotheses 
about prediced outcomes and structural relationships » Guidelines for participation 
must be drawn up, prior to observation, with the cooperation of individuals involved. 



Biases of 0 must be continuously dealt with but this will depend on whether 
they are a legitimate source of error. Where 0 bias will produce variation equal 
to or greater than phenomenolpgical variation , it is necessary to articulate and 
hypothesize about bias x behavior interaction in a manner suggested in general terms 
by Mydral (1953) , Where bias is of minial important (as in many cultural anthropor 
logical studies) it need be only articulated. 



Just as in any data gathering process, inferences are only as strong as the 
instruments that are used, PO depends on high quality O’s who can demonstrate their 
perspicacity by being able to predict Interactions and circumstances and to relate 
observed behavior to given theoretical models. Proof of quality can either be left 
to the readers of final reports or it can be currently brought into relief by using 
multiple d*s with parallel systems. The test of effectiveness or precision is 
clearly not a reliability coefficient or an ratio. Any such statistical test 
works smoothly once the data is obtained and disregarding the validity of the data. 

PO emphasize^ letting meaning speak for itself in much the same way the Skinnerians 
proclaim that data should be directly recorded and then speak for itself. 

The assumption of PO is that there are 0*s and tijethodologies which can be used 
to obtain data that reveals more about observed processes than about 0*s. Method- 
oloeierLHe designed to efficiently utilise O's with given degrees of competing 
biases and with specified goals with reference to designated behavioral systemso This 
id to say that design will have to be adapted for known variations in 0*s, goals and 

systems 0 



PO is not clearly defined methodology that is unifonnly used in the social 
sciences. The practice of having an 0 look closely at a segment of interpersonal 
(or individual) behavior Is simplemihded and elementary. Where more clearly defined 
procedures are appropriate they^^^ould certainly be usedo The designations of 
adequate 0*s is difficult and perhaps, often impossible. It might appear that PO 
is a regression to pre-scientific methodology, where uncontrolled judgements are com- 
bined with unknown weights. But is is even less scientific to lise "powerful” instru- 
ments to perform tasks for which they are unsuited , The decision to use PO is made in 
light of the complexities of teaching, the difficulties of obtaining comparable samples 
of behavior, the problems of irreversibility, the tenuouaness of child behavioral 
crits^rla and the obscurity and ineffectiveness and inappropriateness of personality 
measurement for obtaining adequate measurements of teacher characteristics. This could 
lead to the abandonment of such research or, as in the case of PO, to the adaption of 
relatively crude processes which can, albeit subjectively, deal with those obstacles. 
Developments in audio-visual technology will make it possible to give more substance to 
the inferences of O’s and to provide reasonably direct documentation of classroom 
processes that can be exposed to more varlfled procedures o 
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APPLICATIONS IN HEAD START AND ELEMENTARY S CHOOL CLASSES 

Applications of modifie^articipant observation approaches were made on selected 
^ Head Start and Element^ry'Sbhool classes in connection with two projects, w^^ch were 
taking place concurrently. The first involved twenty Head Start classes which were 
• being evaluated by the Boston University Head Start Evaluation and Research Center as 
a Part of its participation in the National Evaluation Program. The second was with 
Project Concern, an experimental study of the effects of suburban education on inner- 
city children in and around Hartford, Connecticut. Since the data on both of these 
projects with regards to the tested and observed performances of individual children, 
has not been made available, this report is necessarily incomplete. Procedures for 
observing classes and obtaining data will be described in some detail, and preliminary 
descriptive statements will be made with regards to dimensionality of scales that 
were used in each investigation and agreement between raters on a variety of scale 
ratings. In addition, for the Project Concern application, the division of classes 
into inner-city defacto segregated and suburban unite with one, two or three bussed 
negro children in them permits a straight forward comparison over location of classes. 

Although the general principles behind participant observation, as developed by 
Bruyn ( 1966) were followed in the development and carrying out of procedures, the 
sustained and intensive contact of observers with classrooms and schools was not 
followed partly by choice, because of the kinds of variation that were of most interest, 
and partly by necessity. Future studies will provide for considerably more contact 
between observers and the institutions they are observing in order to realize the depth 
which is only being approximated by procedures to be reported herein. 

The alms of these studies were twofold: 1) to study the relationship between 
selected characteristics of teacher style and changes in mental abilities, academic 
achievement, personal-social development and creativity of children in selected 
classrooms; 2) to describe, through cross-sectional procedures, teaching situations 
which Head Start children arr exposed to and those to which they will most probably 
be exposed to if they attend inner-city or suburban elementary classrooms. 

PROCEDURES 

Bo.th applications called for the recruiting and training of observers who had 
extensive experience both as teachers and as observers of preschool and elementary 
- ' . school classes. Initial training sessions involved observation of classes and 

discussion of an all-inclusive categorical model of classroom procedures (Appendix C). 
This model was not for the purpose of providing a checklist or of focusing observers 
attention on particular variables so much as it was for directing their attention to 
all possible contingencies and teaching situations. The model included listings, under 
the genieral heading instruction , of materials, lessons, motivation, evaluation, and 
achievement. A second section under the general heading of controls included form, 
quatitity, tone, consistency and student pressure. Facilities listed characteristics, 
and Implications for teaching. Student interaction included opportunity character- 
istics. A last category, teacher-student interaction . included humor, address, feelings, 
reinfomcement. This model was meant to be a vehicle which would serve to provoke 
discussion and generate questions about varieties of teaching experiences. In addition, 
an exhaustive list of variables associated with teachers, students and curriculum was 
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constructed, through the deliberations of observers, in order to sensitize them to 
differences between independent, intervening and dependent variables (Appendix F) , 
It is critical to note that the models developed from observational seminars and 
were, therfefore, the produce of the efforts of observers. They were not handed 
listings of categories and variables which had been developed externally and whicn 
would have been, therefore, imposed upon themo 



observers were asked to keep detailed notes on tneir observations without 
regard to a particular model, but with specific regard to what they considered 
to be the most important characteristics of the classrooms they were observing. 
These notes were to be transformed into process* reports which were to be concluded 
by analytical reports and summary interpretations (Appendix D) • 



Scales were developed for both studies by observers after carefully and 
deductively describing contrasting characteristics of teaching situations which 
observers judged as being relatively unique. (See appendices A&E) » The scales, 
are, therefore, a reflection of differences seen by observers, rather than the 
basis for making distinctions. This meant that this approach to studying teaching 
involved a concomitant study of observer vd^iation, and that these two separate 
focuses were mutually interdependent. 



The burden of responsibility was clearly on observers rather than scales and 
it called for an inferential process which would be only as defensible as the 
perceptiveness and intelligence of the observers permitted. This process structures 
a systematic approach to dealing with subjective impressions of observers who are re- 
quired to defend these impressions in the face of careful scrutiny by other observers 
and by senior members of the project staff. The process assumes that each observer 
has enough experience and insight to be able to produce salient reports and inter- 
pretations of teaching variation. Resulting inference must attend to both sources 
of variation — teaching and observing — in order to adequately describe stylistic 
variation within stylistic categories. 

In order to provide a superstructure for teaching and observing variatfLo.ns, films 
of selected classes were developed. In covering a wide range of activities, these, 
films have and will continue to provide referent behaviors for the reports and ratings 
of observers. Extensive use of these films has been and is continuing to be made in 
order to clarify reductions of behavior that were made by observers. 



OBSERVATION OF HEAD START CLASSES 

Of the twenty sample classes used in the National Evaluation program, nineteen 
were observed sufficiently by two or more observers to produce reports and ratings 
on a series of scales which were constructed by observers during the course of their 
observations , 

Eight scales were used in rating nineteen teachers by six observers, with each 
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teacher being rated by two, three, or four separate observers. 

The scales were as follows: 

lo Attitude towards teaching situation© 

2o Teachers differentiation of children and activities., 

3o Predominant emphasis of curriculum o 
4o Purposefulness of classroom behavior® 

5o Control bf materials and interactions o 

6o Commtnication-responsiveness 0 
7o Work-play continuumo 
8o Overall rating o 

The detailed statements about each of these scales were given to each observer 
and can be found in Appendix A® 

Rater agreement on the ten scales varied between 80% and 90% and on the_overall 
ratine the agreanent was 92%. Interscale correlations varied between .60 and .90. 
V^lation belween classes appear to be sufficient to allow for maximal rater agreement 
as well as the probable inflation of scale inter-correlation. 

Observers were instructed to sort all teachers on each scale, rather than rating 
each teacher on all scales, in order to minimize halo effects. 

Since four of the six observers had training and experience i“ early childhood ed- 
...arion and cLsequently held a point of view which valued highly differentiated pro- 
grams with a considerable amount of freedom for individual children, “‘J^r 

irrnecessarily a reflection of this point of view and are, therefore limited in their 
generality. Observational teams that participate in such a strategy 
a wide sp^trum of points of view with at least two observers representing each major 
variation. Similarly, it is essential to obtain samples of classes where competence 
and style are relatively independent so that their respective sources of variance can 

be partialed outo 

^>R0JECT CONCERN : Comparisons of inner-city and suburban classes . 

Project Concern is a large scale interventional project which provides for 
educational placement and supportive services for 250 inner-city children. The 
city children are all residents of Hartford, Connecticut, and the experimental inter- 
vention consists of placement in surrounding middle class suburban schools. A randomly 
selLted control group of 250 children is being studied concurrently in order to test 
hypotheses regardiigthe differential effects of inner-city and suburban schools on 
children. A summary of the theoretical framework and the experimental design of 
Project Concern can be found in Appendix B® 

The Boston University Head Start Evaluation and ReseaUch Center has been involvt. ^ 
ift observing and filming a random sample of classes that contain experimental • 
control children® Observations have also been made on a sample of Head Start classes 
so that educational continuity between Head Start and elementary school could be 
ascertained® Filming took place within a careful observational survey design so that 
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tiitt validity of the filming process could be evaluated o 

Prom thirty-n^.ne schools involved in Project Concern, thirty-eight classes were 
aelectocl for the observational and film survey* Teh of these classes were filmed over 
a five-month period* The extent to which filmed behaviors of particular classes repre- 
eent those classes, as well as the extent to which the film classes are representative 
of all classes, is presently under careful consideration. Findings thus far are that 
inciependent observers can go from films to reports and from reports to films with equal 
i^aciiity and that ratings of films are in almost complete agreement with observer 
ratings made of filmed classes at other times during the year* 

Both the observational and film survey Included kindergarten, first, second, 
third I and fifth grades in both inner-city and suburban schools Inner-city classes 
Hrere selected randomly (stratified on grade) from the total pool of control classes. 
Suburban classes were selected randomly from two communities that had greatest partici- 
pation in the project and that represented more and less cooperative coomunities with 
regards to Project Concern* 

The observational team consisted of five observers with widely different back- 
grounds and points of view* They were trained, respectively, in preschool education, 
elementary education, elementary and special education, secondaj^ and special education, 
end elementary education and counseling* Each observer was randomly assigned a sample 
oi^ classes in both inner city and suburban schools* They were required to make at least 
two extended observations, separated in time by at least one week, and, preferably, three 
or four separate observations* In addition, each observer^ was required to observe 
classes of two other observers at least once and, preferably, twice each* 

bbservers wrote process and Interpretive reports and ratdd, each class on ten scales 
that had been derived by the observational team from preliminary observations of the 
total sample of classes. A sorting technique was used so that a given ratpr would 
foeus on inter-class variability over each scale, rather than within class variability 

on all scales* 

Scale derived areas follows: 

1* Involvement and interest of children 

2* Purposeful behavior of class , 

3* Source of direction of academic activities 
4. Nature of control over behavior 
5* Effectiveness of behavioral controls 
6. Quality of presentation of subject and materials 
7* Differentiation of instruction 
8* Teacher reaction to Classroom situation 
9. Reinforcement of behavior of children 
10* Nature of reinforcement 

With the exception of scales five and nine, there appears to be a general factor 
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Which differentiated teaching in observed classes. Intercorrelations between s a es 
ratified between .60 and .80 and the internal consistency of the scales is well documented 
acrLs all observers by scale— total score correlations of .80 and .90 with the exception 
Of the two scales mentioned. Rater agreement on individual scales , with the exception 
o^ scale 9 varied between .50 and .60 and rater agreement.on the cumulative mean rating 
that was made by each observer on each teacher was correlated .65. 

there are important differences between raters as is reflected by their respect- 
ive interscale correlation matrices. For two of the raters, the interscale correlations 
were generally between .40 and .60, while two of the obher observers had interscale 
correlations between .75 and .85. Subsequent data analyses which are aimed at estab- 
lishing differential effects within suburban and inner-city classes will treat observer 
score Ltrices separately in order to access the Validity of different observational 
points of view with respect to predicting change in diverse educational settings. 

Data obtained from scales was unequivical in showing suburban classes to be 
uniform'ly superiot"^to inner-city classes. Seventy-five <ipercent of the suburban classes 
were above the median and seventy percent of the inner city classes were below the 
median which was highly statistically significant on "t" test. 

Differences between inner-city and suburban classes were statistically significant 
on nil scales except 5, effectiveness of control ; 9, reinforcement of behavio£; an , 
the nature of reinforcement . 

Thus, observational ratings clearly distinguish innerrcity and suburban classes 
on selected scales and on mean rating over all scales. However, 30% of the classes over- 
lap, five suburban classes being below the median and six inner-city classes being 

above the median. 



These observational data will be used in order to modify the prediction of change 
in inner-city and suburban classes in order to determine whether high quality (as here 
defined) classes in inner-city schools are associated with changes in children in high 
quality classes in suburban schools and, similarly, whether low quality instruction 
itl the suburbs is associated with low quality instruction in the inner city. 



DISCUSSI ON 

This carefully structured observational survey 'demons tra ted the degree and 
of difference that is manifest be'tween inner-city and suburban classes.^ This is ac e 
up by a film survey of selected classes, kindergarten through five, in inner-city 
and suburban schools. There is a close correspondence between filmed behaviors and 
these that are reported in the data analysis of the scales used by observers. In both 
cases it is apparent that .inner-city schools are characterized by relatively uninvolved 
children, classes with extremely restricted purposes and teachers who tend to per-^ 
vasively control materials and children. This control is often expressed as coercion 
and threats and is accompanied by a rather pedestrian presentation 6f materials with 
relatively little differentiation of instruction. Inner-city teachers appear to enjoy 
their teaching less than suburban teachers. These differences are quite apparent n e 
films, which are presently being prepared for showings at several national conventions. 
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Inner City and suburban classrooms will be displayed simultaneously on two adjacen 
screens in order to bring these comparisons into relief* Films have been su jec e 
to detailed analyses in order to refine scaler differences. Films of the inner city 
and suburban classes have been combined with films of Head Start classes n or er 
to specifically and objectively present a cross sectional longitudinal comparison 
of the experiences that children have in preschool, kindergarten and through the 
grades* The films vividly portray the contrast between selected Head Start and selected 

elementary school classes o 

All filmed sequences have been coded according to a curricular scalematic de- 
vised by Garfunkel (1967) which identifies activities according to curricular 
classification (activity, substantitive or routine), substantitive or activity 
category (construction, performance, play gratification, language, social science, 
snacks, clean up or rest), process focus (mechanistic routine, skill, perceptual, 
cognitive or social) and control (teacher or child dominated). Each sequence is 
Also rated on the scales developed by observers. This allows for matching of con- 
ttasting curricular and stylistic sequence across and within Jobation (inner -city- 
suburban) and grade level (Head Start and Kindergarten through Grade Five). Further- 
more it provides a basis for comparing filmed sequences on ten classes to observed, 
recorded and rated behaviors in 38 classes which were selected by using systematic 
and random s'ampling procedures. The validity of the films is, 

on techniques and methods of selecting classes and filming then, and analytically, 
by obtaining comparable data on films of a limited sample of classes and anecdotal 
repcrts and ratings on a representative sample of both inner-city and suburban classes. 



Preliminary findings from these studies document wide variations across 
Head Start inner-city and suburban classes. The obvious next step is to follow children 
who have been exposed to certain styles of teaching and to compare their responses 
tb elementary schools that offer similar and contrasting classroom environments. This 
can serve as a control for predicting how high and low changes on various measurement 
procedures will respond to continuous and discontinuous learning environments. Of 
particular interest will be the interactions between Head Start and elementary school 
stylistic variations on selected measures of achievement and social-emotional be— 

havlorfi . 
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Appendix A April 1, 1967 

HEADSTART EVALUATION AND RESEARCH CENTER 

Boston University 

Scales for Rating Participant 
Observational Reportd of . 

Headstart Classes 



1. Attitude towards teaching situation 

4 J 

* 

This icble is specifically aimed at a judgement of whether the teacher enjoys 
the teaching situation and not whether she is a good teacher or whether the observer 
likes her* At the high end of this Scale such adjectives as happy, pleased, exhil- 
erated, joyful, and so fouth’. . At the low end of the scale, unhappy, miserable, 
sad, pained, and so forth. The judgement revolves around what the observerrsees in 
the behavior of the teacher and not a projection by the observer as to whether 0 
would be happy doing the things that the teacher ife doing. This, as well as other 
judgements. Will depend upQn evidence that is collected in the course of observa- 
tions,' and it should be possible to sight that evidence. Therefore, it is the- 
oretically assumed that the total behavioral protocal is reducible in such a way as 
to provide bits of evidence to support each scaler judgement. Without such 
te^ucibility, the judgement becomes simply a "gut readtion." While admitting 
that the '!gut reaction” iis.* an important part of perception and judgement, the 
process of collecting evidence and making judgements should force the observer to 
l6ok deeply into his reaction and to make essentially two judgements: the first 

one bfeing Whether or not he can make a rating, and the second being conditional on 
an Affirmative response to this. The condition of being able to make the rating 
will always depend upon the articulation of evidence to support a given judgement. 

2. Teacher's differentiatiation of children and activities 

At the end of the scale we have a teacher who runs a class that has a high 
rating of individual instruction and who does not make demands Upon groups of 
children to do the same things at the same time. High differentiation would involve 
either one of two strategies; A.) where there is a special plan for each child 
depending Upbn his abilities and rttitudes and b.) where each child is allowed to 
go his own way and to 'seek out his* own kind of activity and activity level. Low 
differentiAtion would be evident by a preponderance of classroom activities which 
involve all bhildren. It does not fdUow from this that this scale will necess- 
arily correlUte with good teaching or poor teaching, but that it represents a 
style of teaching with respect tc dealing with individual children of groups of 
children. 

3. Rredomlnfeht Emphasis of Curriculum 

I 

This is Essentially a nominal scale which calls for a judgement on the part 
of the obsCrVEr as to which of the cAtegories suggest the principal manifest goals 
of the activity being observed. The extent to which these categories are ordinally 
related depends upon a presumed value system with regards to desired goals of 
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piesohool teaching. The oategoties to be. used in this scale are taking directions. 
Lgnitive, perceptual, social emotional and a fourth category, unclear, w^ch in 
dicateS that no single emphasis can be inferred from observed activity. The judge 
mehti Of which category a given sequence of behavior belongs to, wi ^ example^^if a 
the behavioral priority system that operates for a ® ® if ^he 

jiiven lessdn or period appears to be dominated by cognitive training 

^eWior of the children'cause changes in plans and ,ies 

then dognitive wculd be viewed as being a secondary goal, and the kind 
t.Tb-ir*h r^oom'tive training give way to would be the primary designation. It is essen 
11^1 tSafwf oLerve cUstel cloLly and long enough so that we can make inf-ences 
about what the goals, in fact, are, rather than what they are said to be. Freeplay 
periods might be dominated by something like the learning of routines and/or 
language tLining. Perceptual training might very well .be dominated by social/ 
emoaonal conspirations If the behavior of the children causes the 
the- emphasis for individual children,- ffoiii.tim6-.to time; .-.As 

the other scales, it will be necessary for observers to present evidence for manifest 
golirald to dishnguish between the nominal categories of this scale and overall 
ilfglmllt If effectiveness. A good deal of work will have to be J®®'® 

to that it presents the observer with a series of branbhing sea ^ 

categories, but with a theoretical connection between the different branches. 

4. Purposefulness of classroom behavior 

An affirmative response to this scale will depend upon clear evidence of 
direction and continuity. One would expect to find a considerable amount of 
observer disagreement over this scale because this is particularly subject t 
Whether or' not the observer is in harmony with the teach.er. and is able je® ^e 
underlying goals of the class as it evolves. In order to rate a teacher as being 
lirposiful Ind the class as being purposeful, it will be necessary to show 
Ividance for continuity and direction; and similarly, inorder for a teacher t 
rated as being not purpohcful, it will be necessary to point out discontinuity 
and to show many apparent shifts in direction during the course of observation. 

S* Control of Materials 

The auestion here is not so much whether it is the child or the teacher but, 
father, whether the child has a say in either the gross selection 

matorials or in their use after they are selected, or whether the teacher dominates 
both selection and •uge'.hc ^ * 

6a Communication--Responsiveness 

This question is directed at the class and raises the issue of whether, 

whatever is going on in the class, there is great '®®P°"®^''®"®®® 

Part of the children or .are they largely unresponsive or indifferent and, if ^"7 
thing, following through on routines rather than being responsive to activities 
and to the teacher. Responsiveness is indicated by a large amount of ®“ 

non*Verbal communication, but it does not indicate that this communication is 
cotTidtructiVO or destructive or that it is good or bad. 






o 
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7». Work and Play 

At the high end of the scale, work and play are undifferentiated and the 
teacher makes little attempt to label or construct activities as being work or 
play, but, rather, they tend to meld together. At the low end of the sc^le there 
is a clear distinction'-certain activities are presented as play activities and 
others are presented as work activities. 

8. This scale is for a total "gut reaction" to the teacher, class, and children; 
and it asks the observer to indicate, without any great demand for evidence, that 
he thinks a given teacher is more or less effective. 

All</of these scales are intended to get at ordinal distinctions between a specified 
sample of teachers that a given observer has been assigned. All judgements are 
necessarily comparative,: arid they will depend upon what observer has seen as a 
part of the observational task. It is the job of the designer of the sample to 
make sure that each observer has a fair distribution of teacher variability in his 
sample and, furthermore, that this variability is not highly skewed. This means 
that the assignment of a sample of classes to a given observer must be proceeded 
by .enough observation to provide evidence for gross variability within a given 
sample of teacher. Samples for observers should have relatively homogeneous- 
variance . 
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Appendix B 
October 10, 1966 

PROJECT CONCERN 

T. W. Mahan, Project Director 

Brief Summary of Theoretical Framework 



PROJECT CONCERNi although directly related to the problem of ^ facto segregation, 
is hot essentially an experiment in integration; rather, it is an experiment in 
educational . intervention designed to aomteract the limited influence of urban 
education on the disadvantaged. Research has described the "cumulative deficit" 
which the child from the low socio-economic enifironment tends to exhibit in his c : 
school performance- -a phenomenon which is dramatically accentuated among the non- 
Vhite poor--and has underlined the profound task involved in reversing the trend. 

A feview of the literature quickly communicates the impression that the problem 
go6s beyond special teaching techniques, enriched materials, and better programming. 

PROJECT CONCERNITwill be evaluated by measured changes in pupil behavior. Nonethe- 
less* it is important to outline, at least in skeletal fashion, the theoretical 
base from which these changes are predicted. Basically, the research stems from 
a cotiviction that changes in stimuli, environment and other input data can result 
in changes in response or output behavior. However, it also felt that cognitive 
patterns fpr copying with formal learning situations and the affective responses 
which accompany these patterns have been well crystallized at the time of school 
entrance. This results in the use of traditional response patterns which, for the 
disadvantaged, are frequently ineffective for school goals. To counteract this 
established tendency it seems best to present the subject with an intense and 
pervasive experience in a fadically different environment so that new responses 
can be provoked. This is the first stage of PROJECT C(^CERN--to create some dis- 
sonance within .^he pupil in terms of his usual perception of himself in relation 
to school and to take advantage of this period of flux by reinforcing positive 
behaviors and attitudes. 

The second aspect of the intervention model is tied to the influence of peers as 
a basis for the development of role fulfilling behaviors. By placing a limited 
number of inner city youth (about 10% of the classroom population) in a suburban 
classroom these same youth will be constantly in contact with models of behavior 
mo 3 ?e in keeping with school values. By limiting the impact of models which 
reinforce the current, ineffective behavior and emphasizing the impact of different, 
but reasonable consistent models, it is hoped that some "shaping" of the pupils' 
learning styles will take place in the direction of increased academic performance. 

I ' 

As a catalyst to prevent too much dissonance which might create a withdrawal 
and/or rejection reaction, significant adult figures who share much of the child's 
heritage but also exhibit the desired characteristics in terms of attitudes toward 
school and learning are provided in the supportive team. The effectiveness of this 
additional factor in the change process is a focus of the research design and, 
hopefully, evidence will be available at the termination of the project to determine 
the dif fetiential liiiipact of the learning environment as separated from the impact of 
adult identification figures. 



Tn essence PROJECT CONCERN focuses around the change in perception, already to a 
large extent stereotyped, which can be accomplished by a confrontation with experi- 
ences highly charged with novelty but also in a context of interpersonal support. 

It is predicted that changes will take place and that they will take 
direction of the models which the suburban youth present to the bussed pupils. 
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EXPERIMENTAL DESIGN 

PROJECT CONCERN is designed to determine the relative effectiveness of a radically 
diffecent educational environment as a preventive and corrective intervention in 
the educatidn of urban youth from the inner city. The theoretical rationale for 
the position has been discussed abovej but the pragmatic aspects must be mentioned 
briefly here. The "vacant seat" for pupil assignment has resulted in considerable 
Variability in the placement with somt; classes having only one experimental S while 
others have four. This in turn has created a situation which results in the experi 
mental Ss b6ing;spreAdoacross thirty-three ;<J33). schools while control Ss are drawn 
from six (6) schools. Hopefully, this diversity will have a seif-cancelling effect 
which will underline the impact of the experimental variable - the treatment pro- 
cedure. In this same regard, it is also important to stress that *the Experimental 
SA not receiving external supportive services are all placed in one school system 
(6 schools) and that generalizations from their performance must be made with that 
fact clearly in mind. 

Nonetheless the design; seems adequate to examine the relative impact of four (4) 
methodologies 5 on- ‘the ’l'ea£>hing 9 -atctcddeif9dandfmotivatJt’ons of-inaef ^-iltycyouth. 

These methodologies, in order of their predicted effectiveness, are as follows: 

1) Placement in a suburban system with supportive team assistance. 

2) Placement in a suburban system without supportive team assistance. 

3) Placement in an inner city school with supportive team assistance. 

4) Placement in an inner city school without supportive team assistance. 

Ss assigned to treatment procedures one (1) and two (2) above are considered to be 
Exparimehtal Ss since they are subject to the impact of the major variable under 
study: placement in a radically different educational environment. Ss assigned 

to treatment procedures three (3) and four (4) above are classified as controls. 

As described above all Ss were drawn from the same population in a random fashion. 
Schematically, the design is as follows: 





Experimental 




Groups 




Control 




Groups 


6lrade 


With Support 


Without Support 


With Support 


Without Support 




N 


Schools 


N 


Schools 


N 


Schools 


•N 


Schools 


Kdg* 


32 


8 


14 


3 


1 . 1 . 


. - 


50 


1 


1 


38 


9 


5 


2 


12 


2 


40 


2 


1 


.47 


9 


2 


2 


12 


2 


40 


2 


3 


30 


7 


7 


3 


12 


2 


40 


2 


4 


25 


6 


9 


4 


12 


2 


40 


2 


5 


41 


9 


6 


2 


““ ““ 


“ — 


40 


1 



IT” 

ERIC 












- 4 - 




The criterion variables which will serve as basis for evaluating the effect of the ^ 

treatment variables (suburban school placement and supportive team assistance) can be | 
grouped into four (4) general headings? 

a) Mental Abilit}' 

1. Wechsler Intelligence Scale for Children 

2. Primary Mental Abilities 

b) Academic Achievement 

1. Reading 

2. Listening 

3. Arithmetic 

c) Personal-Social Development 

Sociometr iccS tatusn 
Test Anxiety 
Attitudes 
Teacher Ratings 
School Attendenee 
Vocational Aspiration 

d) CCreativity 

1. Picture Completion 

2. Circles 

These data will be collected at four points; Septembei^ 1966,.' as a’ base; May, 1967, 
to evaluate effects after one year; September, 1967, to assess loss diiring the summer; 
May, 1968, to evaluate effects after two years. The basic statistical tests to be 

uSed will be analyses of variaiVce'' and covariance. •^>^Allwda.tav(^illbfc§na'nalyzedGfp]^. they 

e thge^olloy? jggpV^f 4:ables with.. thgyp.cimfiryni^a^i^iebleslicagejiaSiejr, r, grande' 1 . 
placement, school system, and where the N permits, school. 

In addition, case study materials reported on a weekly basis by teachers will be 
utilized in an attempt to discover patterns of growth and development. Along with 
this approach there will be data collected which will indicate parental involvement 
and attitude as well as neighborhood reaction to a child's placement in the suburbs. 

It is anticipated that there will he significantly greater growth for the Experi- 
mental Ss as a group, but it is also hoped that evidence as to most productive and 
effective intervention for pupils with differing characteristics may be revealed by 
careful manipulation of the results. 

The techniques described above will be employed on the total samples. However, 
it is expected that smaller samples drawn from these samples will be used to 
study other areas such as speech improvement, frustration tolerance, and personality 
variables. The major outcomes of the Project will be evaluated from this design 
framework by means of the following specific hypotheses stated here as predictions. 

Por operational purposes, a "statistically significant difference" shall be defined 
as a deviation of such magnitude that its likelihood of occurring by chance does 
not exceed one in twenty. 



1 . 

2 . 

3. 

4. 

5. 

6 . 
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1) Experimental Ss will have significantly greater gain scores 

than control Ss in: . 

a) all measures of mental ability 

b) all measures of academic achievement 

c) all measures of cognitive flexibility (creativity) . 

2) Experimental Ss will show significantly greater decrease than 
control Ss in measures of: 

a) general anxiety 

b) test anxiety 

3) Experimental Ss will not differ significantly from eontrol'uSse 
in sociometric measures of: 

a) acceptance by classroom peers 

b) acceptance by neighborhood peers 

4) Analyses of teacher report data on Experimental Ss will show 
a pattern of sequential responses which follows the following 
trend for Ss who show significant gains in academic performance: 
uncritical acceptance by the teacher; more realistic appraisal 
by the teacher, but with a tendency to emphasize assets; a 
tendency to recall and report successes and achievements; 
attainment of a plateau in terms of reporting pupil behavior 

as being relatively unexceptional and consistent. • 
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Examples 


INSTRUCTIONS 




A* Materials 





Characteristics and amount teacher prepared, commercial student 

prepared 

Conteilt--specifically , the amount, nature, or characteristics of 
topics related to urban environments or problems. 



B. Lessons 




Interpretation 


by teacher or student and the amount. 
"What would have happened if there were 
no Civil Way'." 


Deviations within lessons 


Does the teacher allow students to 
introduce or follow issues that may lead 
away from lessons? 


Spontaneity 


Does T. allow asides, immediate student 
reactions, etc. during lessons? 



Opportunity for Participation Does T. call on all students? Do faster 



Individual Participation 


ones dominate? Are slow ones encouragwd 
and given a chance? 

Amount of individual reading, board work, 
participation. 



*What are the project student's reactions during recitation? How much 



participation, attention. 


cooperation? 


C. Motivation 




Origin 


teacher, children, a combination through 
some form of theme. 


Pursuit 

« 


Does T follow children's ideas, accounts 
even fantasies? 


Charac.'^er is tics 


What is discussed? How is the environment 
utilized? 

0 


D. Evaluation-Achievement 




i Type 


tests; oral statements, displays of stUdentd 
works. (Are project students works 
displayed?) 
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Category 

TEACHER-STUDENT INTERACTION 



Examples 



A. Humor 



Does T utilize humor to Include students as 
opposed to ridicule. 



B. Address 



How does she address individuals or the 
class? "Boys and girls." "Students." 
"Children" Last «iames--first names. 



C. Feelings 



Does she express or discuss her own feelings 
and attempts to elicit those of the students? 



D. Reward-Punishment 



How does she express her favor or disfavor. 
"I'm proud of you." "I like obedient 
children." 



*Examples of specific interaction with project students. 
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Category 



Examples 



I DISCIPLINE 

i 

A . Form 

{ 

I Verbal-direct 

1 

! 

Verbal- indirect 



Auditory 
Visual 
physical 
B. Amount 



"Sit down." "Don't do that." 

"Please write the word." "Why don't 
you put your .books away." 

Clapping the hands, striking the piano 

The evil eye 

holding, touching, etc. 

How many discipline instances during ary 
dne visit. 



; ! 




I i 



C. Tone Must the class be completely silent. 

riow much noise is allowed. 

D. Consistency Is the teacher consistent with her rules 

dnd enforcing them*? 

E. Student presstir.e Are there occasions when students discipline 

or assist the teacher in this area by 
bringing pressure upon others. "Sssh, be 
quiot." 



PHYSICAL ORGANIZATION OF CLASSROOM 

A. Characteristics 

B. Room divisions 

C. Interaction 



Straight rows, tables, clusters of two 
and three desks. 

Are their study areas, work areas, hobby 
areas, reading areas, etc. 

Does room organization assist teacher- 
student and student-student interaction. 



^ I 




I ■ 
I 







S^UDeMT INTERACTION 

A. Opportunity Does the seating, ” lessons , and assignments 

allow or encourage interaction. Learning 

• groups, work groups, teacher's assistants. 

$ 

B. Characteristics Describe interactions. Students selecting 

one another to write spelling words on 
board,- or to clean the desks, etc. 

^Degree of project student's "mix." Do they choose others, are they 
aggressive, moderate, or retiring in their interactions. 
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Boston » Massachusetts 

CLASSROOM OBSERVATION AND WRITTEN REPORTS: 
INSTRUCTIONS FOR OBSERVERS 



introduction : 

In order for us to most effectively use your observations of classrooms, It will 
be necessary for us to have several kinds of reports which will reflect. In a variety 
of ways, the teacher and child behaviors which you have observed In the classes 
assigned to youo These reports must be detailed enough and must Include sufficient 
affect so that other readers can read a series of reports and rate them In ways 
similar to ' the ways In which you will be requested to rank and rate the various 
classes that you are observing." This does not call tor the' suppress Ibn of your 
biases, but rather the ready admission of them and explicit attempts to distinguish 
between those behaviors which you take a liking to as differentiated from those 
behaviors which you think are of high quality. This means that you have not only 
to observe and report what you see, but also to assimilate what you see Into the 
.model that Is represented by your Ideas, feelings, and experiences. We 
shall bring together the various models of the several observers into an Integrated 
framework which Is controlled partially by the outline which was distributed and, 
further, by a series of scales which will be presented to you after you have 
concluded your observations. 

The process of abstracting from classroom behaviors' to your observations, and 
then to your written reports and then, still further, to a series of relevant scales 
Is a difficult one which will depend on the kinds and degrees of differences that are 
found between the various classes that you observe. 'Difficulty Is, at the same time, 
a function of the differences that exist within any one class over a period of time. 
The process that Is being constructed will give a more or less clear Indication of 
whether classes are des crib ably and meaningfully different and," to- a lesser extent, 
the degree of differences between these classes. The' reliability of'the process will 

depend upon the clarity and comprehensiveness • of • the' written • reports . It Is 
necessary both to be able to carefully describe the classes'that weseeas well as 
to tnake some clear statements about how equlvlcatl or unequlvlcal the system of 
measurement Is when It Is put to a fair test. In this 'case' the" tests 'will Include 
the observations of classes by different observers • as well as the ratings of the 
classes by Individuals who have not seen them, but who have access to the written 

reports . 
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OUTLINE FOR CLASSROOM OBSERVATION 

This outline, which was distributed to each observer, Is not to be used as a j 

checklist or as an observational gulden Rather, It should be used In the following ^ 

ways observers should read and reread it carefully so that they are quite familiar I 

with the' various categories and sub-categories that describe a more or less all ; I 

inclusive listing of behavioral possibilities in classroom situations » The outline j 
does not represent a mutually exclusive system nor does it cover the detail which 
would bring it so much closer to the classroom situation o Observers should be quite 
familiar with it, but they should not actively use it during the course of their 
observations. After completing process reports, they should refer back to the outline ; 
in order to sensitize, them to the kinds of Information they are getting and the 
behaviors and situations which they should attend to .on future visits to the class. 

The outline will be referred to again when the summary report is discussed below. 



RROCESS REPORTS 

These should include a detailed statement of everything that Is observed in the 
classroom Including the behaviors of the teacher and children, the lihyslcal 
characteristics of the classroom, the materials that are used and any other 
observations which are pertinent to discussing the class. These reports are to 
be thought of as the total of the observer/class interaction and they should not 
exclude the observer and his feelings from the report. 

Observers will differ in the way in which they construct this process report, 
but the end result should be pretty much the same. Some of you will take notes as 
you Are observing the class, others should write out a detailed report immediately 
after you leave the class, still others might develop a system for sketching out their 
dbservatlons so that they can then be transcribed into a running commentary describing 
vbat was seen and how it was seen. 

TheUe process reports are the raw materials for everything that follows and a 
Single report should be made out for every observation of the class. Therefore, each 
observer will have at least two and preferably three process reports on" each class 
that. they observe. 

It is hoped that these reports will not simply be a rather dry chronological 
listing of everything that happens but that they will Include appropriate adjectives 
aUd interpretations that are a part of the observational process. The total 
interpretation of a given teacher and classroom will come in a later report. What 
we are interested in here are the more minute interpretations' of the specific 
behaviors that are observed. Although we are not specifically attending to 
fragmentary quantitative questions such as how many times a given child’ is repri- 
manded or how often the teacher talks opposed to how often the children talk.' But 
we should be quite aware of' duration and quantity and appropriate notes should be 
made about persistent kinds of behaviots that take place. 

The process reports will be used in two waysi in the first place they will be 
used by independent readers who will make judgements about the classes from reading 
these reports; in the second place, they will be used to document the findings of 
this survey and relevant parts of these reports will be abstracted and' Integrated 
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Into a total report of all classes o In both uses of the process reports It Is 
necessary to have writing that Is provocative and comprehensive and that projects 
the reader Into the classroom so that he gets a feeling for what is taking place 
and how It Is taking place » 

ANALYTICAL REPORTS ‘ 

There should be an analytical report for each visit to a classroom. This 
report represents the observer's explanation and synthesis of what he has seen. 

It can draw upon the material from the process report but it Is not an observational 
report as such but rather a critical appraisal of the classroom for the period of 
time that was observed. If there Is no substantial difference between several” 
process reports, It Is possible to combine several of these Into cne analytical 
report. However, In general, there will be a separate analytical report for each 
process report. 

The analytical report should refer back to the outline and should assess which 
parts of the outline are most relevant for the class under consideration, and what 
kinds of Information are not readily obtainable either because of the structure of 
the class or because of the accident of having observed a particular kind of class 
or a particular segment of fhe curricular. 



SUMMARY INTERPRETATION 

There will be one summary interpretation for each teacher that you observe. 

This will draw upon the several process reports and analytical reports and it should 
Integrate all of the material that you have in your possession. This summary report 
should have two sections to it: first, an open-ended judgemental and Inferential 

report describing the essential of the observed behavior of the period of two or 
three observational periods. It should be completely openended (projective). In that 
you are free to draw on any material that you have In any of the visits and you 
should underline freely as you see fit. The second part of the summary report should 
closely follow the outline and should comment on each of Its major sections. If there 
are many omissions here then it should be clear that you have not observed the class 
either a sufficient number of times or sufficiently long enough on any one time. We 
continually have to address ourselves to the question of whether we have observed 
behaviors which make any particular class comparable to other classes. 

Classroom observation Is continually plagued by the lack of comparability of 
data. In one class a teacher may do a large amount of talking and it might be 
considered to be extremely important in assessing her effectiveness. Another teacher 
may also do a lot of talking but it might be trivial compared to .other behaviors 
which she displays In her work with children. Tills means that the problem of 
describing and evaluating teachers has to consider more and less effective behaviors 
as well as behaviors which are not applicable in an assessment of effectiveness. 

Somewhere along the line, we must make judgements which stem from our 
descriptions and which say something meaningful about the degree and kind of Impact 
a particular teacher might have. We must obtain a sufficient amount of material on 
teachers to make judgements about how effective they are with respect to the teaching 
of academic subject matter, of providing an environment for individual self- . 
determination, and encouraging appropriate Inter-personal relationships between the 
teacher and the children and between the children. 
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HEADSTART EVALUATION AND” RESEARCH CENTER 
Boston University 

Scales for Ratine Elementary School” Classes* 
Involvement and interest of children 
Indifference. 


1 

] 

Curiosity, ! 


Apathy 


Absorption 


Puirposeful behavior of class 
Aimless 


Direct 


Wanderlni^ 


Responsive 



Source of direction of academic activities 



Teacher - 

4. Nature of control over behavior 



.Child 

(Teacher) 



6 . 



Coercion 

Threat 



5. Effectiveness of behavioral controls 



None 



Class'-wr nf control 

Quality of presentatlonof subject or* materials 



Pedestrian,. 

Routine 



7* Dlfferentatlon* of' Instruction 



Monolithic,. 

Uniformity 



Teacher reactlonto classroom situation 



Unhappy^ 



Indifferent 



hostile 

Reinforcement of behavior of children 
Not apparen t 



LO. Nature of reinforcement 



Negative,. 

Punitive, 



JBrlbery. 



Threatening 



Trust , 



Respect 



(Teacher) 



.Complete., 

Class well controlled 



(Teacher) 



.Creativity 



Variety , 
Innovation 



(Teacher) 



.Highly differentiated. 



Individually discriminated 
(Teacher) 



Happy I 



Involvement with children. 

Obvious enj( 

(Te 



...Joyment 
[Teacher) 



.Frequent 



(Teacher)” 



.Positive, 



Approval , 
Encouragement 



*A11 teachers observed by a given rater are to be sorted into* five categories so that 
two-thirds of the teachers ‘are* In categories' 2 i3, and;4; one-third are to be 1 and 5. 
Category 1 Is the left hand side of each scale and category 1' to” the right hand side. 
Category 3 Is an Intermediate 'category. 



APPENDIX E* 

VARIABLES FOR OBSERVATIONAL SCHEDULES 
(WITH SELECTED REFERENCES) 

Variable Types 



Obiect 


Independent 

Characteristics 


Dependent 

^ Behavioral 


Curricular 


Pupil 


School background 
Placement procedure, 
2 

Diagnostic Informa- 
tion: 2, Aptitude 
Achievement , 
Personality 
Family-Home 


Problem solving, 2 I 

Motivation, 2 I 

Attention, 1 I 

Curiosity, .1 I 

Activity, 2 I 

Orlglnatldn I 

Mobility 1 

•Participation I 

Disruption I 

Individuality , 2 I 

Pupil-Pupil Inter- 1 

action, 2 I 

Sociometrld variables I 


Grouping, 2 
Getting help, 1 
Independent 
Activitlty, 1,2 


Teacher 


Education 

Experience 

Age 

Sex 

Certification (s) 
Professional Or- 
ganizations and 
Journals 
Attitudes, 3 


Preparation, 2 I 

Direction, 2 I 

Presentation Variety,! I 

Sequence, 2 I 

Verbal-Nonverbal 
Management-Discipline 
Empathy-Support-Humor , 1 
Evaluation-Criticism, 2 
Reinforcement-Rewards, 1 


Use of curriculum 
guide, 2 

1 Textbooks , work- 

books, 2 
Teacher-prepared 
materials , 2 
Evaluation-Reports, 
2 


Pupil- 

Teacher 

Interf- 

action 


Not applicable 


Direction-Initiative, 1 
Social Organization- 
Teacher or pupil cen- 
tered, 1 
Delegatloii of 
responsibility 


1 Differentiation, 

1,2 

1 As related to 

1 content and 

1 procedure 


Classroom 


Demographic 
Location-Type of 
community 
Size 

Equipment 

Supervision- 

reported 

Level 


Supervision-observed 
Climate, 1 
Routines, 2 
Discussion, 2 
Competition, 2 
Order-DlsOrder, 2 


1 Content, 1: 

1 Academic-Vocational- 

1 Crafts-Soclal- 

1 physical and 

1 recreational 

1 Subject or project 

I Consultants-muslc, 

1 art , physical 

1 education, 2 

1 i 



1. Classroom Observation Code Digest (Cornell, Llndrall, Sarpe, 1952) 

2* Schedule for observing special class for mentally retarded ■ children 
(Blatt, 1963) 

Minnesota Teacher Attitude Inventory (Cook, Leeds, and Collis, 1951) 
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